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1. Introduction 


In late 2009, the National Science Foundation (NSF) and the William and Flora Hewlett Foundation 
charged the National Research Council (NRC) with forming an ad hoc committee to examine the 
potential for computer games and simulations to support science education. In response, the NRC’s 
Committee on Science Learning: Computer Games, Simulations and Education, convened a two-day 
workshop to review what is known about the role of computer simulation and gaming in science learning 
and to outline a set of priorities for future research. 


The NRC committee described games and simulations as existing on a continuum and sharing many of 
the same characteristics. They defined simulations as models of complex systems in which users can 
explore the implications of modifying parameters and often allow for dynamic, as opposed to static, 
visualizations.’ Simulations allow users to observe and interact with representations of phenomena that 
would otherwise be sensorily invisible or exist in the abstract. Scientists routinely develop and apply 
simulations to model and understand natural phenomena across a wide range of scales from subatomic to 
planetary. Games can share many of these characteristics, but unlike simulations, are often played 
spontaneously, in informal contexts, and primarily for entertainment. They typically feature explicit rules 
and goals that drive the user experience, allowing users to receive feedback about their progress and their 
actions to affect the state of the game.* The committee distinguished between games played solely for 
entertainment and “serious” games, which are intended (e.g., by the user, classroom teacher, or developer) 
to serve an additional purpose. In STEM, serious games are designed to support the learning process, and 
the committee focused specifically on science games that accurately modelled or simulated scientific 
processes, and where interactions within the game’s virtual world were governed by scientific principles.* 


In their report, the NRC committee concluded that the use of computer games and simulations had great 
potential to affect a variety of science learning goals, such as conceptual understanding, motivation, 
science process skills, science discourse and argumentation, and understanding of the nature of science. 
However, at the time of the report, research on the educational impact of games and simulations had not 
kept pace with the evolution or implementation of these products. This dearth of research made it difficult 
to establish a base of evidence for the effectiveness of computer games and simulations. Thus, the 
committee issued several recommendations targeted at improving research quality, filling gaps in the 
research, and creating partnerships to institutionalize research and development within the field. 


The NSF’s Discovery Research K-12 (DR K-12) program is one potential source of funding for the 
needed research on the educational effectiveness of gaming and simulations. The DR K-12 program seeks 
to enhance the teaching and learning of STEM in K-12 education by funding the “development, testing, 


' Clark, D.B., Nelson, B., Sengupta, P., & D’ Angelo, C. (2009). Rethinking science learning through digital games 
and simulations: Genres, examples, and evidence. Paper commissioned for the National Research Council 
Workshop on Gaming and Simulations, October 6-7, Washington, DC. | Plass, J.L., Homer, B.D., & Hayward, E.O. 
(2009). Design factors for educationally effective animations and simulations. Journal of Computing in Higher 
Education, 21, 31-61. 

* Clark, Nelson, Sengupta & D’ Angelo. (2009) | Hays, R.T. (2005). The effectiveness of instructional games: A 
literature review and discussion. (Technical Report No. 2005-004). Orlando, FL: Naval Air Warfare Center 
Training Systems Division. 

> National Research Council. (2011). Learning science through computer games and simulations. Committee on 
Science Learning: Computer Games, Simulations, and Education, M. A. Honey & M L. Hilton (Eds.), Board on 
Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National 
Academies Press. 
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deployment, effectiveness, and/or scale-up of innovative resources, models, and tools”* in STEM areas. 
This work takes on many forms within the funding portfolio, including computer games and simulations. 
With the growing attention to these types of educational technologies, the Community for Advancing 
Discovery Research in Education (CADRE)”’ conducted a brief targeted study to better understand the 
gaming and simulation research and development work being funded within the NSF’s DR K-12 program 
and to highlight work that may help fill the gaps identified by the NRC. 


The remainder of this document describes a set of innovative projects that include a gaming or simulation 
component from the NSF’s DR K-12 program. It begins with a description of the methodology used in 
this targeted study, and then describes findings related to the targeted users, educational resources 
developed for users, research methods, and sustainability. (See popups for more information.) 


2. Methodology 


CADRE annually compiles a descriptive overview of the characteristics of projects in the DR K-12 
portfolio. ° In addition, CADRE also conducts targeted studies on topics of interest to DR K-12 principal 
investigators (PI), the NSF, and the broader research community. The current targeted study was designed 
to provide insight, using selected DR K-12 projects as a set of illustrative cases, into the innovative 
research and development being conducted with educational games and simulations. The research 
questions, data, and analytic methods used in the study are described below. 


2.1. Research Questions 


At the time of publication, the NRC’s Learning Science Through Computer Games and Simulations’, 
reported that the evidentiary base for learning science from simulations was stronger than that of games, 
and overall the research on the efficacy of games to promote learning remained inconclusive, partially due 
to the relative newness of serious games compared to simulations. The committee recommended a 
research agenda for simulations and games that aimed to improve the overall quality of research as well 
as advance the field of research and development around games and simulations for learning. Some of the 
outlined approaches include: 


- Examining how different contexts (i.e., formal contexts such as K-12 science classrooms or 
informal contexts such as after-school clubs) affect science learning 

e Specifying clearly the desired learning outcomes of the game or simulation 

- Assessing the potential of games and simulations to advance learning goals, including motivation, 
conceptual understanding, science process skills, and scientific discourse 

e Applying the evidence-centered design approach to the development of assessments through 
games and simulations 


“ National Science Foundation. (2011). Discovery Research K-12 (DR K-12): Program Solicitation 11-588. 

° CADRE is the resource network that was established by the NSF to support the DR K-12 community in advancing 
the state of research and evaluation in STEM education. Organizational partners of CADRE include Education 
Development Center, Inc., Abt Associates Inc., and Policy Studies Associates. 

° The latest portfolio review can be found at 
http://cadrek12.org/sites/default/files/CADRE%20YR%205%20Portfolio%200verview%20v7_stl.pdf 

” National Research Council. (2011). Learning science through computer games and simulations. Committee on 
Science Learning: Computer Games, Simulations, and Education, M. A. Honey & M L. Hilton (Eds.), Board on 
Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National 
Academies Press. 
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e Developing and using statistical models and machine learning approaches to make meaning from 
and patterns of student behavior using the large amounts of data collected through games and 
simulations 

- Establishing alternative mechanisms for supporting large scale collaborative innovation and 
ongoing improvement as well as exploring systems for informing users about the quality and 
effectiveness of games and simulations for learning 


The conceptual base for this study was guided by the NRC report along with suggestions from active 

DR K-12 PIs in this field. It is important to note that since the NRC’s 2011 report, which provided an 
overview of the field prior to 2010, there have been several advancements pertaining to games and 
simulations research as the field continues to adapt, evolve, and advance. Recent literature reviews have 
found positive impacts associated with learning using computer-based and serious games, though more 
research is needed on the role of contextual and instructional factors. ® The research questions guiding the 
analysis were: 


1. What is the target audience of the gaming and simulation components of these projects? 
How are the projects distributed across different disciplines? 

3. For what contexts are these materials developed? 

What resources and products are these projects developing? 

What research designs and methodologies are being implemented in these projects? 

a. What outcomes are captured in these projects and how are they measured? 

b. What are the methodologies used by the projects to assess the effectiveness of the educational 
gaming and simulation components? 

6. How are the projects addressing dissemination and sustainability? 


as 


2.2. Data 


The sources of data used in this study were collected by CADRE, directly from DR K-12 PIs, for the 
purpose of preparing annual portfolio overviews of the DR K-12 program. Each year, CADRE has asked 
Pls of newly funded projects to provide CADRE with their projects’ proposals and responses to review 
panel questions. Additionally, CADRE asked Pls of existing projects funded in prior years to provide 
project updates in the form of annual reports, project publications, and other information about their 
plans, activities, and achievements. For the current study, 66 projects were originally identified from the 
portfolio review’’ as those involving a broader range of technologies, including educational gaming, 
interactive learning, and/or virtual environments. This sample was then reviewed by CADRE staff and 
gaming special interest group (SIG) members to identify the projects that include a significant gaming or 
simulation component and/or directly address gaming or simulation for K-12 STEM learning. A total of 
33 projects were removed from the sample, resulting in the final set of 33 projects. 


8 Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of the 
cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249.) | Connolly, T. 
M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic literature review of empirical 
evidence on computer games and serious games. Computers & Education, 59(2), 661-686. 

° CADRE operates under a cooperative agreement (rather than a contract) with the NSF so we do not have access to 
the data and materials maintained by the NSF. 

'© This study included projects receiving new awards between 2007 and 2012. Three additional projects (NSF 
Award Nos. 1221614, 1119290, 1119383) met the inclusion criteria but are not represented in the final set due to 
insufficient data. Brief descriptions of these projects are included in the Appendix. 
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Note that, for convenience, we refer to these projects generally as “gaming” projects in the text that 
follows. This is not to discount the differences between games and simulations or to suggest that the same 
questions, priorities, and methods are guiding the research and development work being done in these 
areas. Rather, it reflects the fact that their comparison was beyond the scope of this report. Limited data 
about the projects constrained the ability to classify the projects cleanly at various points along a 
continuum differentiating between simulations and games. 


2.3. Coding Methodology 


This study drew on two types of coded data: (1) general project characteristics that were coded for the 
annual portfolio overview and (2) more in-depth coding conducted specifically for this study. 


To generate data on general project characteristics, as part of CADRE’s annual activities, the collected 
DR K-12 project materials were reviewed and coded by a team of CADRE researchers using a protocol 
designed to capture information on project attributes and characteristics as well as the DR K-12 program 
goals being addressed. 


More comprehensive coding, specific to the research questions of this targeted study, was then conducted 
on the subset of gaming projects. After developing supplementary coding schemes to further characterize 
projects’ methodologies, measured outcomes, and analysis techniques, two coders reviewed and coded 
the available project materials, including proposals and annual reports. After coding for the presence or 
absence of a particular quantitative methodology, the second level involved identifying the type of 
methodology used for data analysis in a project. The results are presented and elaborated in the findings 
section, accompanied by illustrative examples from selected projects. 


2.4 Study Limitations 


The materials reviewed and coded for this analysis were created by grantees for purposes other than this 
review. Thus, across projects, the information is reported in diverse and unsystematic ways. As a result, 
the level of detail reliably extracted and coded varied and was, at times, limited. 


Additionally, project materials often reported on planned activities. For convenience, this report uses 
phrases like “projects used,” but the study’s data sources contained information on what projects 
proposed to use or discussed considering in their annual reports. While CADRE’s request for additional 
materials included responses to NSF reviewers, annual reports, final reports, publications, and products, 
there were no materials beyond the proposal for 13 projects. 


An additional limitation is that the insight gained through this study is limited to the set of DR K-12 
projects with a gaming or simulation component, and it is unclear how representative projects funded by 
DR K-12 are of the larger work being conducted on educational gaming and simulations. While the 
findings cannot be generalized to the wider field of K-12 educational games and simulations, they do 
provide a snapshot of the current and recent research and development being conducted on games and 
simulations used in education within the DR K-12 program. 


3. Findings 


The NRC report’s definition of games and simulations as well as its proposed research agenda described 
provided guidance for this study. The findings in this section, organized by research question, present 
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descriptions of the disciplines covered, context, outcomes, data collection methods, data analysis 
methods, and strategies for dissemination and sustainability employed in these projects. 


3.1 Disciplinary Content 


While each of the STEM disciplines was targeted in the DR K-12 gaming portfolio, the major discipline 
addressed was science. Twenty-one out of the 33 projects focused exclusively on science, five focused 
exclusively on mathematics, and only two had a focus on engineering. Five projects addressed multiple 
disciplines as shown in Table 1. 


Table 1: Major Disciplines Addressed 


Number 


(N=33) Percentage 
Science only 2 64 
Mathematics only 15 
Engineering only 6 
Multi-discipline’ 15 


Science and mathematics|(2) science and social studies[(1} and science, engineering, and computer science|(2)} 


Physical Science in Elementary Grades: 


The Leonardo Project: An Intelligent Cyberlearning System for Interactive 
Scientific Modeling in Elementary Science Education 


This project designed and implemented technologies that combined artificial 
intelligence in the form of intelligent tutoring systems with multimedia interfaces to 
support children in grades 4-5 learning science. The students used LEONARDO's 
intelligent virtual science notebooks to create and experiment with interactive 
models of physical phenomena. With this technology, students’ models 'came alive' 
as interactive multimedia artifacts that combine animation, sound, and narration. 
The curricular focus was on physical and earth sciences, and the technology 
supported multimodal interactive scientific modeling for four curricular units: 
forces and motion, magnetism and electricity, landforms, and weather and climate. 
A central feature of this environment was PadMates, which are intelligent virtual 
tutors that support science learning through interactive scientific modeling. 


3.2. Target Audience 


While most investigated student outcomes (as described later in this report), all of the DR K-12 gaming 
projects engaged in research activities involving students and teachers. Even if the project materials and 
activities targeted students, they were accompanied by resources for teachers or professional development 
components. Table 2 outlines the number of gaming projects that targeted students in each grade band, 
with some targeting multiple grade bands. Two-thirds of the projects (66%) in the gaming portfolio 
targeted middle school students followed by more than a third (39%) targeting high school students. 
There were only two projects targeting students at the preschool level. 


(as © 2014 CADRE 5 


Gaming in Pre-K: 
Next Generation Preschool Math (NGPM) 


One of two projects implementing games in pre-k, NGPM, focused on preschool 
classrooms in low-income communities where children currently have few 
opportunities to engage with coherent sequences of mathematics instruction or with 
interactive media. A team of learning scientists and media researchers collaborated 
with educational media producers to develop, test, and refine a curriculum 
supplement that promotes children’s understanding of number (counting, 
comparing, and ordering) and fair sharing (equipartitioning) using self-leveling 
games, collaborative games, an open-ended math tool sandbox, and playground and 
snacktime activities. 


Table 2: Grade Levels 


Number 
(N=33) Percentage 


Pre-kindergarten 6 
Elementary school (K-5) 18 
Middle school (6-8) 66 
High school (9-12) 39 
Other 6 


Projects may target multiple grade bands. 


3.3. Context 


The DR K-12 gaming projects involved a broad range of populations and were implemented in both 
formal and informal settings. Table 3 describes the projects’ settings in terms of whether the projects were 
designed and implemented in the classroom (formal school settings) or in after-school clubs, museums, or 
libraries (informal settings). 


Table 3: Project Context 


Number 
(N=33) Percentage 


Formal setting 97 
Informal setting 12 
Other (with parents and community) 6 


Projects may work across multiple settings. 


The NRC report"’ recommended the investigation of how learning in informal contexts may be leveraged 
for learning in the science classroom. The following four projects implemented in informal settings. 
- Leveling Up: Supporting and Measuring High School STEM Knowledge Building in Social 
Digital Games 
e Arcadia: The Next Generation—Transforming STEM Learning Through Transmedia Games 
¢ GeniVille: Exploring the Intersection of School and Social Media 
e CyberSTEM: Making Discovery Visible Through Digital Games Conference 


" National Research Council. (2011). Learning science through computer games and simulations. Committee on 
Science Learning: Computer Games, Simulations, and Education, M. A. Honey & M L. Hilton (Eds.), Board on 
Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National 
Academies Press. 


Bridging Formal and Informal Contexts: 


Leveling Up: Supporting and Measuring High School STEM Knowledge 
Building in Social Digital Games 


This project designed, developed and tested a digital gaming environment for 
high school students that fostered and measured science learning within 
alternate reality games about saving Earth's ecosystems. Players worked 
together to solve scientific challenges using a broad range of tools including a 
centralized web-based gaming site and social networking tools, along with 
handheld smart-phones, and an avatar-based massively multiplayer online 
environment (MMO). The game required players to contribute to a scientific 
knowledge building community; and players rated each other's contributions 
for their value to the communities’ learning and decision-making in solving the 
challenge. Designers worked with high-school teachers to develop bridge 
activities that leveraged science learning in games for use in formal education. 


3.4 Products for Teachers and Students 


The portfolio of gaming projects developed a range of products as supports for its users: students and 
teachers. |Thirty-onel|of the 33 projects developed products to support student learning. The student 


learning materials ranged from full curricula to standalone assessments. In addition, [19 of the 33 projects 
had professional development (PD) programs with materials for preservice and/or inservice teachers. 
Projects designing resources for students had PD components and/or embedded teacher training to 
prepare teachers to use the student resources. Many of the gaming projects incorporated more than one 
activity within their scope or had multiple products as part of their deliverables. Projects might have 
designed an assessment, for example, in addition to a PD plan for teachers. Tables 4 and 5 list the PD 
products and content of the PD programs that were delivered to teachers. Table 6 illustrates the kinds of 
student assessment materials the gaming projects created as part of their activities 


Table 4: Student Learning and Teacher Professional Development Products 
Number 
Student Learning Product (N=31) Percentage 
Computer or internet activities and resources 100 


Curriculum 4? 


Number 
Professional Development Product (N=19) Percentage 
Stand-alone instruction, manuals, guides, or other information 79 


One or two sessions, classes, or meetings 32 


Full professional development course 26 


Networks or learning communities 16 


[6] 
Curricula for course 16 


Supervision or mentoring 10 


Projects can develop multiple student learning and/or professional development products. 
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Table 5: Content of Professional Development Programs/Products 


Number 


(N=19) Percentage 


Student curricular frameworks 63 
Lead specific activities 21 
STEM content 21 
Focus on how students learn [1] 5 
Modeling instructional practice p) 


Projects’ professional development programs/products can focus on multiple content categories. 


Curriculum as a PD Product: 

CyberSTEM: Making Discovery Visual Through Digital Games 
This project developed and tested an integrated gaming platform consisting of five 
model games that could form the basis of integrated game-based curricula. Each 
game came with an associated curriculum in life sciences that teachers, museums, 
and other science educators can use to educate the public about cutting-edge 
science. The project sought to understand if participation in CyberSTEM leads to 
increased learning in six areas: interest in science, conceptual knowledge, scientific 
reasoning, reflection on knowing, participating in science, and identifying as a 
scientist. 


Table 6: Student Assessment Products 


Number 


(N=12) Percentage 


Assessment in a curriculum 58 
Stand-alone assessments 33 
Other assessments 8 


Games-based Student Assessments: 


SAVE Science: Situated Virtual Assessments Using Virtual Environments 
This collaborative research project pioneered new ways of measuring student 
achievement by designing and implementing an innovative model for assessment of 
learning in STEM. In SAVE Science, game-based assessment modules evaluated 
middle grade students’ science content and inquiry skills. Using a database of 
student interactions in a virtual environment, evolving patterns of scientific 
understanding among students were captured and analyzed. 


3.5 Project Data Collection and Analysis 
3.5.1 Data Sources For Gaming Projects 


The gaming projects in the DR K-12 portfolio used a diverse set of data collection instruments and 
methods as shown in Table 7. The data collected were both quantitative and qualitative in nature. Twenty- 
four of the 33 projects (73%) used student tests and assessments (e.g., pretests/posttests), while an almost 
equal percentage (76%) conducted interviews with students. 


Sixteen projects (48%) collected survey data from students, which had items related to self-efficacy, 
attitudes, beliefs, etc. These survey data were quantitatively analyzed using several techniques, discussed 
later in this report. Eighteen projects (55%) gathered students’ usage data from computer-based games for 
analyses. The students’ assessment data were from large-scale science and/or mathematics assessments 


like the National Assessment of Educational Progress (NAEP) and Trends in International Mathematics 
and Science Study (TIMSS). These assessments are used for national and international comparisons of 
student academic achievement.[Seventeen|percent of the 24 projects collecting student test/assessment 
data used NAEP tests. 


Table 7: Data Sources 


Number 


(N=33)* Percentage 
Interviews 76 
Student tests/assessments 73 
Observations 67 
Computer usage 55 
Surveys [16] 48 
Artifact review 21 
Focus groups 21 
Journal entries/diaries 15 


Projects may collect and analyze data from multiple sources. 
For one project, there was insufficient information to identify the data source(s). 


3.5.2. Outcomes Measured 


Projects in the gaming portfolio researched a wide range of outcomes that could benefit the education 
community (see Tables 8 and 9). Twenty-nine projects included a focus on student-related 
outcomes, while 15 assessed teacher-related outcomes. The teacher and student outcomes are 
discussed in detail below. 


A. Student Outcomes 

Twenty-nine of the 32'* projects (93%) in the gaming portfolio assessed the student outcomes domain. 
Twenty-seven of these 29 projects (84%) measured variables in the cognitive domain, such as student 
performance and achievement. Fifty-five percent measured psychosocial variables such as student attitudes, 
career interests, and motivational and emotional attributes. Projects used variables and scales that 
assessed students’ attitudes related to science, technology, games and assessments, achievement-related 
beliefs, and motivational constructs including self-efficacy, goal orientation, and test anxiety and/or 
interest in STEM careers. Sixteen projects (55%) measured other kinds of student behavior (e.g., 
completion of homework, test preparation, cooperative work, attendance, and discipline violations). A 
listing of the instruments used by projects in the DR K-12 gaming portfolio to measure student outcomes 
follows Table 8. 


Table 8: Student Outcomes Assessed 


Number 


(N=29) Percentage 


Achievement/Performance 93 
Attitudes/Beliefs 55 
Student behavior 55 


Projects can address multiple student outcomes. 


" For one project, there was insufficient information to determine the outcome(s) assessed. 
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Instruments Measuring Student Outcomes in DR K-12 Gaming Projects 
Mathematics Topic Assessments 

Research-Based Early Math Assessment (REMA)* 
Science Topic Assessments 

Science Attitudes, Skills and Knowledge Survey (SASKS) 

Cornell Scientific Inquiry Series 

Force Concept Inventory (FCD 

Measures of Academic Progress for Science (MAPS) 
Mathematics and Science Topic Assessments 

National Assessment of Educational Progress (NAEP)* 

Trends in International Mathematics and Science Study (TIMSS)* 

New Mexico Standards Based Assessment (NMSBA)* 

California Standards Test (CST)* 

New York State Regents Exam (REGENTS)* 

Pennsylvania System of School Assessments (PSSA)* 

Utah Performance Assessment System for Students (U-PASS)* 
Psychosocial Assessments 

Attitudes Towards Science in School Assessment (ATSSA) 

Achievement Goals Questionnaire (AGQ) 

Children’s Self-Efficacy Scale 

Early Adolescent Temperament Questionnaire Revised (EATQ-R) 

Instructional Material Motivational Survey (IMMS) 

Junior Metacognitive Awareness Inventory (Jr. MAD 

Motivated Strategies for Learning Questionnaire (MSLQ) 

Patterns of Adaptive Learning Scales (PALS) 

Self-Description Questionnaire IT (SDQHU) 

Self-Efficacy in Technology and Science (SETS) 

Sources of Science Self-Efficacy Scale (SSSE) 

Strong Interest Inventory 

Views about Science Survey (VASS) 


*These instruments represent some of the current large scale and state-level 
assessment measures for science and/or math (capturing content knowledge and 
reasoning skills) that were used as assessments within projects. 


B. Teacher Outcomes 

Fifteen of the 33" projects in the DR K-12 gaming portfolio assessed teacher outcomes. Across this set, 
projects measured various teacher domains including attitudes/beliefs, classroom instruction and teacher 
content knowledge (both disciplinary and pedagogical). Of the projects measuring teacher outcomes, only 
[fouridentified the specific instruments used to assess these variables (see below). Some instruments 
measure multiple constructs, for example, instructional practices, social aspects of the classroom, 
emotional support exhibited by the teacher, and teacher-student interactions. 


Instruments Measuring Teacher Outcomes in DR K-12 Gaming Projects: 
Early Mathematics Classroom Observation (EMCO) 

Mathematical Knowledge for Teaching (MKT) 

Principles of Scientific Inquiry-Teacher (PSI-T) 

Reformed Teacher Observation Protocol (RTOP) 

Stages of Concern Questionnaire (SoCQ) 

Technology Use in Science Instruction (TUSI]) 


'S For one project, there was insufficient information to determine the outcome(s) assessed. 
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Table 9: Teacher Outcomes Assessed 


Number 
15) Percentage 
Attitudes/Beliefs 73 
Classroom practices [8 | 53 
Pedagogical content knowledge (PCK) [3 | 20 


STEM content knowledge 20 


Projects can assess multiple teacher outcomes. 


Measuring Teacher Outcomes: 


Cyber-enabled Learning: Digital Natives in Integrated Scientific Inquiry 
Classrooms 


This project investigated the professional development (PD) needed to make 
teachers comfortable teaching with the kinds of multi-user simulations and 
communication technologies that students use. The key technological resource used 
in the project was the Opensimulator 3D application Server (OpenSim), an open 
source, modular, expandable platform used to create simulated 3D spaces with 
customizable terrain, weather and physics. A comprehensive PD program of over 
240 hours, along with follow-up, was used to determine how teachers could be 
supported to use information and communication technology tools effectively in 
classroom instruction to create meaningful learning experiences for students. 

The project measured various teacher domains including teacher attitudes/beliefs, 
classroom instruction, and pedagogical content knowledge. The PD related 
outcomes were measured using two instruments — the RTOP and Technology Use 
in Science Instruction (TUSI). 


3.5.3 Analysis Techniques 


With respect to the quantitative methods and techniques employed for data analyses in these projects, 
79% used various statistical techniques to analyze survey and assessment data. Statistical modeling 
techniques include ANOVAs, ANCOVAs, multilevel modeling (MLM), structural equation modeling 
(SEM) and hierarchical linear modeling (HLM). Projects in the gaming portfolio also engaged techniques 
such as educational data mining, principal components analysis, and epistemic network analysis to 
construct meaning from complex game data. 


Table 10: Data Analysis Techniques 


Number 

(N=33)" Percentage 
Statistical models (e.g., HLM, MLM, ANOVA) 79 
Data mining 6 
Bayesian networks 3 
Evidence-centered design 3 
Epistemic network analysis 3 


Projects can utilize multiple analysis techniques. 
For four projects, there was insufficient information to identify the data analysis technique(s). 


There has been a recent shift among education researchers to go beyond demonstrating significant gains 


on achievement test performance to understanding how to better support learning for different kinds of 
learners. In their report, the NRC committee highlighted the need for research on effective assessment of 
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student learning and its use to impact learning processes. '* They discussed how achieving the potential of 
simulations and games for assessment and learning would require research and development in all areas 
of assessment: development, implementation, and evaluation. Particularly, they emphasized the need for 
an evidence-centered design (ECD) approach to the development of assessments of learning through 
games and simulations. The ECD" approach involves a process in which the game-designers articulate at 
the outset, the competencies to be measured, the evidence that would constitute that a learner possesses 
those competencies, and the tasks that can be used to draw that evidence. It is worth noting that only one 
project, Change Thinking for Global Science (described below), out of the 33 projects proposed to use the 
ECD approach to assessment development. 


Evidence-centered Design Approach: 


Change Thinking for Global Science: Fostering and Evaluating Inquiry 
Thinking about the Ecological Impacts of Climate Change 


This project aimed to provide clear empirical information (e.g., growth spurts, 
growth plateaus, achievement) associated with middle and high school students’ 
learning of complex and dynamic science related to global climate change. In 
addition to creating learning progressions, curricular units, and assessment 
instruments, the project developed and evaluated ecological simulation and 
modeling resources to accompany the curricular units. Their assessment design 
followed principles of evidence-centered design towards the creation of pretests, 
posttests, and embedded assessments calibrated on the same scale. Pre- and 
posttests were designed to evaluate students’ content and inquiry reasoning over a 
range of complexities and contexts. Additionally, Item Response Theory (IRT) was 
used to create graded response models to investigate scale properties and to score 
student pre- and posttests in the respective measures. 


3.6 Sustainability and Scalability 


The research agenda proposed by the NRC committee emphasizes the need for better understandings of 
design features that support uptake in the marketplace, partnerships that support innovations in science 
education using games, and strategies for disseminating information on the quality and effectiveness of 
games and simulations. This section describes the ways in which gaming projects in the DR K-12 
portfolio approach dissemination and sustainability. Although almost all of the projects included in this 
study identified a plan for dissemination in the documents reviewed, the level of detail related to that 
plan varied, as shown in Table 11.[Thirty-one|projects identified vehicles for dissemination, which are 
described in Table 12. 


'* National Research Council. (2011). Learning science through computer games and simulations. Committee on 
Science Learning: Computer Games, Simulations, and Education. M. A. Honey & M L. Hilton (Eds.), Board on 
Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC. 

'’ U.S. Department of Education, Office of Educational Technology, Expanding Evidence Approaches for Learning 
in a Digital World, Washington, DC, 2013. 
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Table 11: Strategies for Dissemination and Sustainability 


Number 


(N=31) Percentage 
Identified what will be disseminated (ideas and products) 94 
Identified dissemination partners (e.g., networks and associations, opinion 72 
leaders) 
Identified potential adopter or end user 71 
Included potential adopter or end user input in design/development/research 45 
Addressed strategies for sustainability after the grant 16 


Table 12: Dissemination Vehicles 


Number 


(N=31) Percentage 


Presentations/Poster sessions 90 
Journal articles 87 
Academic journals 52 
Practitioner journals 32 
Journals not specified [9 | 29 
Websites 71 
Professional networks 32 
Workshops 32 
Newsletters 23 
Commercial products or publications 16 
CDs/DVDs 13 
Popular media 10 
Reports (not articles or books) 10 
Social media 10 
Blogs 6 
Books/Book chapters 3 
Webinars 3 
White or working papers 3 


Projects can disseminate using multiple vehicles. 


This report describes the characteristics of a subset of projects within the DR K-12 portfolio that included 
a gaming or simulation component. It draws on data from 33 funded projects. The summary below 
addresses a few of the key research questions outlined at the beginning of the report. 


What populations are these projects targeting and how are they distributed across disciplines and 
contexts ? 


The gaming portfolio included projects working in pre-k through high school, with most projects working 
at the elementary, middle, and high school levels (18, 66, and 39% respectively). Seventy-nine percent 
address science topics and 21% focus on math. Thirty-one of the overall 33 projects are involved with the 
studies in the classroom and four of them are situated within informal contexts. 
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What types of educational resources were being developed and researched? 


Projects in the DR K-12 gaming portfolio were developing and researching a wide variety of resources. 
Almost all of the projects developed products targeting students, and more than half developed resources 
for teachers. Slightly more than one third of the projects were designing student assessments as their 
products. 


What are the teacher and student outcomes measured within these projects? 


Both student and teacher outcomes were investigated in the research conducted across DR K-12 gaming 
projects. Eighty nine percent of the projects in the portfolio were researching student outcomes, nearly all 
of them focused on student achievement and performance, and 45% of the projects were researching 
teacher outcomes, with the majority looking at teacher attitudes and beliefs about their practice. 


What are the prevalent methodologies and data collection techniques ? 


In the DR K-12 gaming portfolio, interviews (76%) and observations (67%) were common qualitative 
data sources while assessments and tests (73%) formed the major source of the quantitative data, followed 
by computer usage data (48%). A majority of projects used a mixed methods research design. Seventy- 
nine percent of the projects conducted research using various statistical techniques. Two projects 
employed data mining techniques for their data analysis, one of which used the evidence-centered design 
framework. 


What strategies are projects using to disseminate and sustain their ideas and products? 


Most projects in the DR K-12 gaming portfolio (97%) provided some level of information on their plans 
for dissemination, while only 16% addressed strategies for sustainability in the materials reviewed. 
Projects are using a variety of vehicles to disseminate their products and ideas, with the most common 
being presentations/posters (90%), journal articles (87%), and websites (71%). 
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5. Appendix: Summaries of DR K-12 Gaming Projects 


Summaries are organized by the last name of the PI, which is followed by the NSF award number, project 
title, and a brief description. "° 


Asbell-Clarke — 1119144 

Leveling Up: Supporting and Measuring High School STEM Knowledge Building in Social Digital 
Games 

CADRE Project Profile | NSF Project Summary 

This project designs, develops and tests a digital gaming environment for high school students that fosters 
and measures science learning within alternate reality games about saving Earth's ecosystems. Players 
work together to solve scientific challenges using a broad range of tools including a centralized web- 
based gaming site and social networking tools, along with handheld smart-phones, and an avatar-based 
massively multiplayer online environment. The game requires players to contribute to a scientific 
knowledge building community. 


Asbell-Clarke — 1134919 

Arcadia: The Next Generation — Transforming STEM Learning through Transmedia Games 

CADRE Project Profile | NSF Project Summary 

This project studies the design features of an experimental gaming environment called Arcadia: The Next 
Generation. Researchers are working with a group of formal and informal educators to study the 
connections between scientific inquiry in Arcadia and STEM learning. The project provides a dynamic 
and evolving place where gamers, educators, parents, and citizen scientists can come together to share, 
rate, and build knowledge through a variety of fun science inquiry games. 


Asbell-Clarke — 1252709 

FUN: A Finland US Network for Engagement and STEM Learning in Games 

CADRE Project Profile | NSF Project Summary 

As part of a SAVI, researchers from the U.S. and from Finland are collaborating on investigating the 
relationships between engagement and learning in STEM transmedia games. The project involves two 
intensive, five-day workshops to identify new measurement instruments to be integrated into each other's 
research and development work. The major research question is: To what degree learners in the two 
cultures respond similarly or differently to the STEM learning games? 


Bulgren — 1019842 

The Evidence Games: Collaborative Games Engaging Middle School Students in the Evaluation of 
Scientific Evidence 

CADRE Project Profile | NSF Project Summary 

This project develops a series of interactive on-line games and investigates the effect these games have on 
increasing middle school science students’ and teachers' knowledge and skills of scientific argumentation. 
There are four areas of argumentation addressed by the games: (1) understanding a claim, (2) judging the 
evidence about a claim based on type and quality (objectivity, reliability or validity), (3) analyzing the 
reasoning applied to the claim, and (4) evaluating the claim. 


16 . ° 7 . A i : 
The projects described are at varying stages of completion, but for convenience, all summaries are written in 
present tense. 


(a © 2014 CADRE 15 


Campbell — 1020086 

Collaborative Research: Cyber-enabled Learning: Digital Natives in Integrated Scientific Inquiry 
Classrooms 

CADRE Project Profile | NSF Project Summary 

This project explores the potential of information and communications technologies (ICT) as cognitive 
tools for engaging students in scientific inquiry and for enhancing teacher learning. A comprehensive 
professional development program of over 240 hours, along with follow-up is used to determine how 
teachers can be supported to use ICT tools effectively in classroom instruction to create meaningful 
learning experiences for students, reduce the gap between formal and informal learning, and improve 
student learning outcomes. 


Clark — 0822370 

Scaffolding Understanding by Redesigning Games for Education (SURGE) 

CADRE Project Profile | NSF Project Summary 

This project is focusing on the redesign of popular commercial video games to support students’ 
understanding of Newtonian mechanics. In support of this goal, SURGE develops and implements design 
principles for game-based learning environments, integrating research on conceptual change, cognitive 
processing-based design, and socio-cognitive scripting. These enhanced games bridge the gap between 
student learning in non-formal game environments and the formalized knowledge structures learned in 
school by leveraging and integrating the strengths of each. 


Clark — 1119290 

Enhancing Games with Assessment and Metacognitive Emphases (EGAME) 

CADRE Project Profile | NSF Project Summary 

This development and research project designs, develops, and tests a digital game-based learning 
environment for supporting, assessing and analyzing middle school students’ conceptual knowledge in 
learning physics, specifically Newtonian mechanics. This research integrates work from prior findings to 
develop a new methodology to engage students in deep learning while diagnosing and scaffolding the 
learning of Newtonian mechanics. 


Dede — 1118530 

EcoMobile: Blended Real and Virtual Immersive Experiences for Learning Complex Causality and 
Ecosystems Science 

CADRE Project Profile | NSF Project Summary 

Researchers are studying whether middle school instruction about ecosystem science can be made more 
engaging and effective by combining immersion experiences in virtual ecosystems with immersion 
experiences in real ecosystems infused with virtual resources. Project personnel are developing a set of 
learning resources for deployment by mobile broadband devices that provide students with virtual access 
to information and simulations while working in the field. 


Dunleavy — 0822302 

Radford Outdoor Augmented Reality (ROAR) Project: Immersive Participatory Augmented Reality 
Simulations for Teaching and Learning Science 

CADRE Project Profile | NSF Project Summary 

This project anticipates the needs of learners in 10 years by developing and testing two learning 
simulations that are immersive, interactive, and participatory and use augmented reality in the outdoors. 
Students work in teams to investigate phenomena and solve problems in a gaming environment using 
wireless handheld GPS units. Using a design-based, mixed-methods approach, the researchers examine 
the relationships among augmented reality, learning in science, socio-emotional outcomes, and the 
demographic characteristics of rural, underserved students. 
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Enyedy — 0733218 

Semiotic Pivots and Activity Spaces for Elementary Science 

CADRE Project Profile | NSF Project Summary 

This project employs sensing technologies to help transform students' physical actions during play into a 
set of symbolic (computer) representations in a physics simulation and to engage the children in a 
developmentally appropriate and powerful form of scientific modeling. The students are in grades K—1 at 
UCLA's elementary school, and the intervention is based on the existing content unit on Force and 
Motion. 


Evans — 1118571 

Gateways to Algebraic Motivation, Engagement and Success (GAMES): Supporting and Assessing 
Fraction Proficiency with Game-Based, Mobile Applications and Devices 

CADRE Project Profile | NSF Project Summary 

This project is designing digital games for middle school students that will help them prepare for success 
in Algebra. The games are intended to help students gain a deep understanding of measurement and 
fraction concepts that are critical as they begin to learn algebra. The project studies students’ development 
of fraction concepts, their engagement in the tasks, and the use of hand-held devices as a useful platform 
for games. 


Finzer — 0918735 

Data Games: Tools and Materials for Learning Data Modeling 

CADRE Project Profile | NSF Project Summary 

The Data Games project is developing software and curriculum materials in which data generated by 
students playing computer games form the raw material for mathematics classroom activities. Students 
play a short video game, analyze the game data, develop improved strategies, and test their strategies in 
another round of the game. 


Hacker — 0821965 

Simulation and Modeling in Technology Education (SMTE) 

CADRE Project Profile | NSF Project Summary 

This project develops and researches the academic potential of a hybrid instructional model that infuses 
computer simulations, modeling, and educational gaming into middle school technology education 
programs. These prototypical materials use 3-D simulations and educational gaming to support students’ 
learning of STEM content and skills through developing solutions to design challenges. 


Horwitz — 0822213 

Evolution Readiness: A Modeling Approach 

CADRE Project Profile | NSF Project Summary 

This project uses computer-based models of interacting organisms and their environments to support a 
learning progression leading to an appreciation of the theory of evolution and evidence that supports it. 
The project has created a research-based curriculum centered on progressively complex models that 
exhibit emergent behavior. The project will help improve the teaching of complex scientific topics and 
provide a reliable means of directly assessing students’ conceptual understanding and inquiry skills. 


Horwitz — 1238625 

GeniVille: Exploring the Intersection of School and Social Media 

CADRE Project Profile | NSF Project Summary 

This project examines the design principles by which computer-based science learning experiences for 
students designed for classroom use can be integrated into virtual worlds that leverage students' learning 
of science in an informal and collaborative online environment. GeniVille is the integration of Geniverse, 
an education based game that develops middle school students’ understanding of genetics with Whyville, 
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an educational virtual word in which students can engage in a wide variety of science activities and 
games. 


Johnson-Glenberg — 1020367 

Embodied STEM Learning Across Technology-based Learning Environments 

CADRE Project Profile | NSF Project Summary 

This project conducts interdisciplinary research to advance understanding of embodied learning as it 
applies to STEM topics across a range of current technology-based learning environments (e.g., desktop 
simulations, interactive whiteboards, and 3D interactive environments). The project has two central 
research questions: How are student knowledge gains impacted by the degree of embodied learning and to 
what extent do the affordances of different technology-based learning environments constrain or support 
embodied learning for STEM topics? 


Kafai — 1238172 

Transforming STEM Competitions into Collaboratives: Developing eCrafting Collabs for Learning with 
Electronic Textiles 

CADRE Project Profile | NSF Project Summary 

This project supports the development of technological fluency and understanding of STEM concepts 
through the implementation of design collaboratives that use eCrafting Collabs as the medium within 
which to work with middle and high school students, parents and the community. They examine how 
youth at ages 10-16 and families in schools, clubs, museums and community groups learn together how to 
create e-textile artifacts that incorporate embedded computers, sensors and actuators. 


Ketelhut — 0822308 

SAVE Science: Situated Assessment using Virtual Environments for Science Content and Inquiry 
CADRE Project Profile | NSF Project Summary 

The SAVE Science project is creating an innovative system using immersive virtual environments for 
evaluating learning in science, consistent with research- and policy-based recommendations for science 
learning focused around the big ideas of science content and inquiry for middle school years. Motivation 
for this comes not only from best practices as outlined in the National Science Education Standards and 
AAAS' Project 2061, but also from the declining interest and confidence of today's student in science. 


Klopfer — 1019228 

DRK12-Biograph: Graphical Programming for Constructing Complex Systems Understanding in Biology 
CADRE Project Profile | NSF Project Summary 

This project investigates how complex systems concepts supported by innovative curricular resources, 
technology applications and a comprehensive research and development structure can assist student 
learning in the domain of biology by providing a unifying theme across scales of time and space. The 
project seeks to address four areas of critical need in STEM education: biological sciences, complex 
systems, computational modeling, and equal access for all. 


Kulm — 1020132 

Preservice Teachers Knowledge for Teaching Algebra for Equity in the Middle Grades 

CADRE Project Profile | NSF Project Summary 

This project is using Second Life and other technology to structure carefully planned learning experiences 
for pre-service teachers. Virtual technologies are used to provide pre-service teachers practice in 
presenting and assessing problem solving activities in a virtual classroom with diverse populations. 
Researchers hypothesize that technology enriched strategies have the potential to deepen pre-service 
teachers' understanding and effectiveness in teaching emerging algebra concepts to diverse student 
populations. 
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Lester — 0822200 

Developing Science Problem-Solving Skills and Engagement Through Intelligent Game-Based Learning 
Environments 

CADRE Project Profile | NSF Project Summary 

The project draws upon intelligent tutoring and narrative-centered learning technologies to produce a 
suite of intelligent game-based learning environments for upper elementary school science students. The 
games explicitly model student knowledge and problem solving and dynamically customize feedback, 
advice, and explanation as appropriate. Unlike its predecessor, the platform is multi-user so it can support 
collaboration; offer dynamically generated feedback, advice, and explanation; and provide a pedagogical 
dashboard that generates student progress reports. 


Lester — 1020229 

The Leonardo Project: An Intelligent Cyberlearning System for Interactive Scientific Modeling in 
Elementary Science Education 

CADRE Project Profile | NSF Project Summary 

This project designs and implements technologies that combine artificial intelligence in the form of 
intelligent tutoring systems with multimedia interfaces (i.e., an electronic science notebook and virtual 
labs) to support children in grades 4-5 learning science. The students use LEONARDO's intelligent 
virtual science notebooks to create and experiment with interactive models of physical phenomena. 


Lewis Presser — 1119118 

Next Generation Preschool Math 

CADRE Project Profile | NSF Project Summary 

This project is developing, testing, and refining a curriculum supplement (a hands-on technology) that (1) 
promotes children’s’ understanding of number (counting, comparing, and ordering) and fair sharing 
(equipartitioning); (2) uses interactive media on an emerging handheld platform (touch screen tablets), 
integrating new multi-touch activities with existing hands-on activities; (3) enhances opportunities for 
learning with interactive media through shared use with adult guides and peers; and (4) provides 
professional and technical support materials for preschool educators. 


Millar — 1119383 

CyberSTEM: Making Discovery Visible through Digital Games 

CADRE Project Profile | NSF Project Summary 

CyberSTEM is developing and testing an integrated digital gaming network that spans homes, schools, 
and informal learning settings, offering a suite of digital games based on cutting-edge discoveries in the 
life sciences. The project asks if participation in CyberSTEM leads to increased learning in six areas: 
interest in science, conceptual knowledge, scientific reasoning, reflection on knowing, participating in 
science, and identifying as a scientist. The target audience includes youth in grades 6-9. 


Mokros — 0733264 

The GENIQUEST (GENomics Inquiry through QUantitative Trait Loci Exploration with SAIL 
Technology): Bringing STEM Data to High School Classrooms 

CADRE Project Profile | NSF Project Summary 

This project is developing and testing a website, software application, and supplemental instructional 
materials that use publicly accessible genomics data to foster scientific inquiry among high schools 
students. Outcomes for students and teachers include developing knowledge, skills, and understandings 
related to genetic inheritance; data investigation and analysis; the process of scientific inquiry; and 
collaboration. 
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Pedersen — 0628264 

Engaging Middle School Students in Student-directed Inquiry through Virtual Environments for Learning 
CADRE Project Profile | NSF Project Summary 

This project is developing five web-based modules for middle school science that engage students in 
student-directed inquiry and provide teachers with professional development in facilitating this inquiry. 
These modules immerse students in virtual environments for learning (VELs) where they take on the role 
of scientists engaged in a complex task. The virtual settings presented in the VELs support students in 
designing and carrying out their own investigations. 


Perkins — 1020362 

Expanding PhET Interactive Science Simulations to Grades 4-8: A Research-based Approach 

CADRE Project Profile | NSF Project Summary 

Colorado’s PhET project and Stanford’s AAALab are developing and studying learning from interactive 
simulations designed for middle school science classrooms. Products include 35 interactive sims with 
related support materials freely available from the PhET website; new technologies to collect real-time 
data on student use of sims; and guidelines for the development and use of sims for this age population. 
The team will also publish research on how students learn from sims. 


Quellmalz — 0733345 

Calipers I: Using Simulations to Assess Complex Science Learning 

CADRE Project Profile | NSF Project Summary 

This project is pioneering simulation-based assessments of model-based science learning and inquiry 
practices for middle school physical and life science systems. The assessment suites include curriculum- 
embedded, formative assessments that provide immediate, individualized feedback and graduated 
coaching with supporting reflection activities as well as summative end-of-unit benchmark assessments. 
The project is documenting the instructional benefits, feasibility, utility, and technical quality of the 
assessments with over 7,000 students and 80 teachers in four states. 


Quellmalz — 1221614 

SimScientists Assessments: Physical Science Links 

CADRE Project Profile | NSF Project Summary 

The goal of this project is to develop and validate a middle school physical science assessment strand 
composed of four suites of simulation-based assessments for integrating into balanced (use of multiple 
measures), large-scale accountability science testing systems. It builds on the design templates, technical 
infrastructure, and evidence of the technical quality, feasibility, and instructional utility of the NSF- 
funded Calipers II project. The evaluation plan addresses both formative and summative aspects. 


Reichsman — 0918642 

The NextBio Project: A Student Collaboratory for Biology Cyberlearning 

CADRE Project Profile | NSF Project Summary 

This project addresses biology teachers and students at the high school level, responding to the 
exponential increases occurring in biology knowledge today and the need for students to understand the 
experimental basis behind biology concepts. The project studies the feasibility of engaging students in an 
environment where they can learn firsthand how science knowledge develops in the fields of 
bioinformatics and DNA science by performing collaborative, simulated experiments to solve open-ended 
problems. 
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Shaffer — 0918409 

AutoMentor: Virtual Mentoring and Assessment in Computer Games for STEM Learning 

CADRE Project Profile | NSF Project Summary 

This project is developing a system for producing automated professional mentoring while students play 
computer games based on STEM professions. The project explores a specific hypothesis about STEM 
mentoring: A sociocultural model as the basis of an automated tutoring system can provide a 
computational model of participation in a community of practice, which produces effective professional 
feedback from non-player characters ina STEM learning game. 


Songer — 0918590 

Change Thinking for Global Science: Fostering and Evaluating Inquiry Thinking About the Ecological 
Impacts of Climate Change 

CADRE Project Profile | NSF Project Summary 

This project draws from the expertise of a fully collaborative educator-scientist team to create learning 
progressions, curricular units and assessment instruments towards large scale research on the teaching and 
learning of climate change and impacts by 7-12th graders in primarily under-resourced schools. Products 
include eight week curricular units, [PCC-compliant simplified future scenarios, an online interface with 
guided predictive distribution modeling, and research results. 


Squire — 0746348 

Scientific Role-Playing Games for 21st-Century Citizenship 

CADRE Project Profile | NSF Project Summary 

This project investigates the potential of online role-playing games for scientific literacy through the 
iterative design and research of Saving Lake Wingra, an online role-playing game around a controversial 
development project in an urban area. Saving Lake Wingra positions players as ecologists, department of 
natural resources officials, or journalists investigating a rash of health problems at a local lake, and then 
creating and debating solutions. 


Squire — 1258679 

CyberSTEM: Making Discovery Visible Through Digital Games Conference 

CADRE Project Profile | NSF Project Summary 

This workshop addresses the need to connect a range of experts involved in game development and 
research to develop and disseminate best practices. The workshop will also establish a network hub where 
educators and developers can find tools for implementing game-based curricula. The project will bring 
together approximately 100 early contributors, including researchers, teachers, game designers and 
publishers, to inform the next phases of research, development, and production in the field of games and 
learning. 


Wiburg — 0918794 

Math Snacks: Addressing Gaps in Conceptual Mathematics Understanding with Innovative Media 
CADRE Project Profile | NSF Project Summary 

This project is developing and evaluating effectiveness of 15 - 20 short computer mediated animations 
and games that are designed to: (1) increase students’ conceptual understanding in especially problematic 
topics of middle grades mathematics; and (2) increase students’ mathematics process skills with a focus 
on capabilities to think and talk mathematically. 
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Xie — 0918449 

Enhancing Engineering Education with Computational Thinking 

CADRE Project Profile | NSF Project Summary 

This project investigates the educational value of computer technologies for learning engineering. The 
project engages high school students to design, build, and evaluate an energy-efficient model house with 
the aid of computer simulation and design tools. 
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